Skip to content

Add OSSF Scorecard callee workflow#113

Open
ehaldane-digicatapult wants to merge 4 commits into
mainfrom
feature/add_ossf_scorecard_callee_workflow
Open

Add OSSF Scorecard callee workflow#113
ehaldane-digicatapult wants to merge 4 commits into
mainfrom
feature/add_ossf_scorecard_callee_workflow

Conversation

@ehaldane-digicatapult

Copy link
Copy Markdown
Contributor

Pull Request

Checklist

  • Have you read Digital Catapult's Code of Conduct?
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have made corresponding changes to the documentation.
  • My changes generate no new warnings.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.

PR Type

  • Feature
  • Documentation Update

Linked tickets

ENG-312

High level description

This request is to add a callee workflow for OpenSSF Scorecard, to generate appraisals against a repository and optionally publish the results (with a badge). This new workflow splits into two parts, the first using the upstream ossf/scorecard-action, which isn't customisable and uses all checks and policies by default. Conversely, the second part is customisable, taking an input that defines dynamically which policies are in scope and which checks actually run. Publishing the customised results isn't feasible with the CLI alone, so I've skipped that. Results are uploaded as artefacts or to the Advanced Security dashbaord in either SARIF or JSON format.

@ehaldane-digicatapult ehaldane-digicatapult requested a review from a team as a code owner July 13, 2026 12:51

@dblane-digicatapult dblane-digicatapult left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Local review of the OSSF Scorecard callee, cross-checked against the live run in digicatapult/hello-world#25 (both scorecard jobs pass there, but only because hello-world is public). Thanks for the thorough README and the fail-fast validation step. Requesting changes on one blocking item plus a few robustness/consistency points.

Blocking

  • The analysis job permissions omit contents: read / actions: read / pull-requests: read. The hello-world run confirms the job executes with only Metadata: read + SecurityEvents: write, which breaks checkout on private callers and silently degrades several checks on public ones. See inline.

Public vs private

  • The file claims public/private support but has no visibility detection or skip, and publish_results/badge are public-only. Decide one way and make the code match (inline on line 3).

Should fix

  • repo_token declared as both an input and a secret; the input is dead and contradicts the README table.
  • required: true + default: on results_file/results_format.
  • Inputs interpolated directly into the docker run: block.
  • checkout@v6.0.2 / upload-artifact@v7.0.1 pin drift vs the rest of the repo.
  • No guard against results_format: json + upload_type: dashboard.

Nits (non-blocking)

  • examples/generate-security-scorecard.md:9 typo: "default mod" -> "mode".
  • Custom-scan passes both --checks and --policy; the generated policy already disables the unselected checks (and uses score: 1, so it enforces nothing meaningful), so the two are redundant. Confirmed non-fatal against scorecard's options.go.
  • ENABLE_SARIF: ... || '' still registers the env var, so scorecard treats SARIF as enabled even for json (its LookupEnv check returns set). Harmless, but not what it looks like.
  • upload_type vocabulary (dashboard/artifact) diverges from the other security workflows (sarif/artefact); an unrecognised value silently uploads nothing.

@@ -0,0 +1,159 @@
name: Scorecard analysis workflow

# Only use on public/private repositories where Code Scanning has been enabled.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This says public and private, but nothing in the workflow detects repo visibility or skips accordingly, and the permissions block below only works for public repos today. Two coherent options:

  • If private is genuinely in scope: fix the permissions (see the comment on the permissions: block) so private callers actually work.
  • If this is public-only in practice (publish_results / badge / REST API are all public-only per the README note): guard it, e.g. if: ${{ !github.event.repository.private }} on the job, or fail fast, and reword this line.

As written it is neither: it advertises private support it cannot deliver, with no guard to skip private callers.

required: false
inputs:
results_file:
required: true

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

required: true combined with a default: is contradictory for workflow_call inputs: required forces the caller to pass a value, so the default is unreachable, yet the README lists these as Required with a default. Set required: false (the defaults are sensible) or drop the defaults. Same applies to results_format just below.

required: false
type: string
default: "archive" # archive or git only
repo_token:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

repo_token is declared here as an input and as a secret (lines 6-8). Every step reads secrets.repo_token; this input is never referenced. The hello-world run shows it riding along empty in the Inputs dump while the real token arrived via secrets:. The README Inputs table documents it as a with: input, so a caller who follows the table will have it silently ignored. Drop this input, keep the secret, and move the README row into a Secrets section.

analysis:
name: Scorecard analysis
runs-on: ubuntu-latest
permissions:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: this token is missing contents: read (plus actions: read, pull-requests: read).

Because this block is an explicit allow-list, the token that reaches analysis is exactly these two scopes. Confirmed in the hello-world run (digicatapult/hello-world#25), where the job's own token dump reads only:

Metadata: read
SecurityEvents: write

Consequences:

  1. Checkout code clones with this token, so on a private caller repo it fails outright. Public repos clone unauthenticated, which is the only reason hello-world stayed green.
  2. Even on public repos, several requested checks (Branch-Protection, Code-Review, Token-Permissions, Signed-Releases) need contents/pull-requests read to evaluate, so they silently degrade while the job still exits 0, giving a falsely reassuring pass.

This also contradicts the repo's own guidance in README.md:41 (private callers need contents: read, actions: read, pull-requests: read).

Suggested:

    permissions:
      security-events: write
      id-token: write
      contents: read
      actions: read
      pull-requests: read

and mirror it in the two example callers and the README permissions table.

-w /workspace \
"ghcr.io/ossf/scorecard:${{ inputs.scorecard_version }}" \
--repo="github.com/${{ github.repository }}" \
--checks="${{ inputs.custom_checks }}" \

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

${{ inputs.custom_checks }} here (and results_file, file_mode, scorecard_version, github.repository elsewhere in this block, plus line 104) is expanded into the script text before bash runs, which is the script-injection pattern our CLAUDE.md calls out (never interpolate into shell commands). Callers are trusted so the blast radius is bounded, but this assembles a docker run command line, so please pass these via env: and reference "$CUSTOM_CHECKS" etc., exactly as you already do for GITHUB_AUTH_TOKEN / ENABLE_SARIF.


steps:
- name: "Checkout code"
uses: actions/checkout@v6.0.2

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pin drift from the rest of the repo: every other workflow is on actions/checkout@v7, this is a major version behind on @v6.0.2. actions/upload-artifact@v7.0.1 (line 146) is also pinned to a patch tag where the repo uses @v7. Please align both with the repo convention.


# Upload the results to GitHub's code scanning dashboard.
# Commenting out will disable upload of results to your repo's Code Scanning dashboard.
- name: "Upload to code-scanning"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

github/codeql-action/upload-sarif requires a SARIF file, but nothing prevents results_format: json + upload_type: dashboard, which would fail at this step. Relatedly, the artifact step above hardcodes name: SARIF file even for JSON output. Consider validating (json implies artifact-only) or documenting the constraint, and making the artifact name format-aware. Only the sarif+artifact path was exercised in the hello-world run, so this combination is currently untested.

Comment thread README.md

| Access | Jobs used | Level | Reason | Conditions |
| ------------------------ | ---------- | -------- | ----------------------------------------------------------------- | ---------- |
| `security-events: write` | `analysis` | Workflow | To POST new code scanning alerts based on the SARIF report | N/A |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This table omits contents: read (and actions: read / pull-requests: read), which private callers need per the repo's own note at README.md:41 and which the analysis job requires for checkout and for several of the checks. Please add them here and in the two example callers once the workflow permissions block is fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants